Interpreting Dynamic Scenes by a Physics Engine and Bottom-Up Visual Cues

Humans have the remarkable ability to infer many different physical properties of objects from observing dynamic scenes. Here, we propose a generative model for inferring physical object properties from real world videos of objects splashing in the water. At the core of our generative model is a 3D physics engine which models the object’s interactions with the environment based on parameters such as density, surface area, shape, and volume. We infer these properties through the combination of bottom-up visual cues and a MCMC algorithm, which drives the physical simulation to match real world observations. Results show that our model makes accurate estimations of unobserved physical properties such as mass and density.